Foot measuring and sizing application

ABSTRACT

Systems and processes for measuring and sizing a foot are provided. In one exemplary process, at least one image of a foot and a horizontal reference object from a first point of view is captured. A portion of the foot may be disposed against a vertical reference object. The at least one image may be displayed, where one or more camera guides are overlaid on the at least one displayed image. In response to aligning the one or more camera guides with one or more of the horizontal reference object and the vertical reference object, a measurement of the foot based on the at least one captured image from the first point of view is determined.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/796,457, entitled “FOOT MEASURING AND SIZING APPLICATION,” filed onOct. 27, 2017, now abandoned, which is a continuation-in-part of U.S.patent application Ser. No. 15/699,861, entitled “FOOT MEASURING ANDSIZING APPLICATION,” filed on Sep. 8, 2017, and now granted as U.S. Pat.No. 10,420,397, which claims priority to U.S. Provisional Ser. No.62/434,151, entitled “FOOT MEASURING AND SIZING APPLICATION,” filed onDec. 14, 2016. The content of each of these applications is herebyincorporated by reference in their entireties for all purposes.

FIELD

The present invention relates generally to foot-sizing applications.More particularly, and in one aspect, the invention relates to a footmeasuring application running on a mobile computing device with anintegrated camera that can quickly and accurately inform the user thedimensions of the foot being measured through an image capture process,and give accurate and real-time feedback during the measuring process.

BACKGROUND

Traditionally, the size of a foot is determined by a physical device,such as a “Brannock” device, which is common in many footwear stores. ABrannock device includes a sliding element to line up with the forwardedge of a toe, where a heel of the foot is braced against an oppositeend of the device. This same device can also measure foot width with aseparate sliding element.

One advantage of these traditional devices is that the user, or thoseaccompanying the user, can see the dimensions generated in real-time.These individuals can agree with the measurements, dispute them, attemptfurther adjustment of the sliding elements or foot position, andtherefore, adjust the foot measurements.

Without a physical measuring device, traditional foot measuring involvesa process of trial and error to determine an appropriate shoe size. Thisprocess involves obtaining shoes of varying sizes so that the customercan try on each shoe and determine an optimal fit. This traditionalmethod depends on feedback from the user in reaction to differentphysical shoes. Such trial and error can be especially challenging withchildren, or those who have physical disabilities or other impairmentsthat make it challenging to gauge accurate feedback from the user.

Furthermore, a less common method of physical size determination is theuse of special shoes with the forefoot constructed from clear ortransparent materials. This method allows for the toe area to be viewedthrough the shoe. Similarly, the goal with this method is real-timefeedback for the user to gauge accurate foot size. However, the maindeficiency is the requirement that all different shoe styles and sizesinclude clear or transparent materials.

As consumers continue to move to online shopping for shoes, there is agrowing interest in alternate methods of foot-sizing that are notdependent on such traditional methods of physical devices or shoes. Onealternative method includes providing directions to a user to trace anoutline of a target foot on a piece of paper and choose two points onthe resulting contour to generate a linear measurement. Problems withthis method include demanding too much time and effort from the user,dependency on the accuracy of the traced contour, and user understandingof how to measure the resulting contour itself. Furthermore, manyconsumers do not wish to be bothered by this process, may draw aninaccurate outline (e.g., by drawing at an angle instead of vertical),or may incorrectly measure the traced outline (e.g., by not measuringalong the major longitudinal axis of the foot).

Another alternative method includes using an application on“multi-touch” device having a relatively large screen, such as a tabletcomputer. These applications attempt to measure foot size by having theuser place the target foot on the screen, while the tablet is situatedon the ground with the screen facing up. Problems with this methodinclude inability to measure feet that are larger than the screen,inability to capture features which are not at ground/screen level andthat affect the measured maximum size (for example, a heel that juts outhigher on the foot), and user sensitivity to device damage by steppingon the device.

Another alternative method includes an application that attempts tomeasure foot size by utilizing a camera built into the computing device.These applications may use a reference object of a known size (e.g., acoin) in the same field of view and obtain a picture of the foot alongwith the reference object. Problems with these applications includeinaccuracy based on flawed processing techniques and algorithms,unacceptably long processing times if the image or multiple images aresent to a remote server for processing, and the lack of real-timefeedback to the user.

Another class of alternative methods include applications which attemptto scan an entire foot and record 3D features of the foot, or byconstructing a 3D digital model of the foot through multiple images. Inaddition to suffering from even longer processing times than the abovemethods due to the greater amount of data in a 3D calculation, there aredoubts about the accuracy of the data generated given the complexity ofthe full-foot 3D model and reliance on in-situ consumer electronics andsettings (e.g., lighting, background differences, etc.). This 3D-basedmethod is also impractical given the limited commercially viable ways toconvert a full 3D digital map of a foot to a completely custom shoe.Three-dimensional printing of shoes mapped to a custom 3D database isnot yet viable as a mass-consumer option due to a number of issues,including the availability of shoe materials that are also available for3D printing (e.g., specific materials for flexibility and durability),and the processing time needed to print an object the size of a shoe.

Therefore, a need exists in the field for a foot measuring applicationthat can both provide real-time feedback to a user and provide anaccurate determination of foot size in a timely and easy fashion.

SUMMARY

According to one aspect of the present invention, a novel foot measuringapplication is described. The application generally involves a softwareapplication program running on a computing device that includes abuilt-in camera, or is coupled to a camera. The computing device may becapable of generating a “live view” of the image field of the camerawhen the camera function is turned on, also referred to as a “previewscreen” or a “digital viewfinder.” The computing device may also becapable of connecting to remote server computers, through TCP/IP orother internet protocols. The novel foot measuring application maygenerate dimensional data by comparing points on the target foot with aknown measuring reference that is also in the camera field of view. Insome embodiments, the application utilizes a sheet of standard-sizedpaper, such as Letter or A4, as a reference object. For example, theapplication may instruct a user to place a sheet of paper on the ground,place his or her foot on the paper (with or without socks or nylons),and then to use the built-in or coupled camera with the applicationrunning on the computing device to capture video (a series of images) ofthe target foot in the viewfinder within a prescribed zone of distancefrom the foot.

Certain embodiments may display an “augmented reality” indication ofmeasurement in the preview screen, such as zones of different colorscorresponding to different shoe sizes, overlaid under or around thetarget foot. For example, such indications may be provided in order togive the user feedback about the dimensions being generated by theapplication's size-analyzing process. These measurement overlays are“target relative” in that they appear to be fixed relative to the targetfoot and paper reference, even if the camera or computing device aremoved (e.g., if the overlays moved when the computing device moved, theoverlays would be defined as “device relative”). Such embodiments betterreplicate the experience of current traditional physical foot measuringmethods. Specifically, the user may remeasure the foot based on theuser's perception of the process accuracy, and understand during thelive process how and why the resulting foot size measurement wasgenerated.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary process for determining measurement of afoot based on at least one captured image of a foot and a horizontalreference object.

FIG. 2 illustrates a screen view of the application, viewable by theuser, depicting a first step in the measurement process and thealignment guides for the reference object.

FIG. 3 illustrates a visualization of the application process, depictingthe measurement algorithm processes of corner and vertex detection,reference establishment, and perspective transformation.

FIG. 4 illustrates a visualization of the application process, depictinga measurement algorithm of target foot detection.

FIG. 5 illustrates a visualization of the application process, depictinga measurement algorithm of target foot boundary determination, rotationdetection and correction, and object measurement.

FIG. 6 illustrates a screen view of the application, depicting themeasurement analysis result that may be viewable by a user.

FIGS. 7A-7D illustrate a measurement process implemented by the footmeasuring and sizing application.

FIG. 8 illustrates a diagram depicting a local client computing deviceand a remote server computer utilized during a measurement process.

FIG. 9 illustrates a screen view of the foot measuring and sizingapplication including an alternate reference object.

FIG. 10 illustrates an exemplary process for determining measurement ofa foot based on at least one captured image of a vertical referenceobject.

FIG. 11 illustrates an alternate embodiment including a computing devicepositioned without a horizontal reference object.

FIG. 12 illustrates a screen view of the application, viewable by theuser, depicting a first step in the measurement process and referencedots.

DETAILED DESCRIPTION

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the term “and/or” includes any and all combinations of oneor more of the associated listed items. As used herein, the singularforms “a,” “an,” and “the” are intended to include the plural forms aswell as the singular forms, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, steps, operations, elements, components, and/or groupsthereof.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by onehaving ordinary skill in the art to which this invention belongs. Itwill be further understood that terms, such as those defined in commonlyused dictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art and thepresent disclosure and will not be interpreted in an idealized or overlyformal sense unless expressly so defined.

In describing the invention, it will be understood that a number oftechniques and steps are disclosed. Each of these has individual benefitand each can also be used in conjunction with one or more, or in somecases all, of the other disclosed techniques. Accordingly, for the sakeof clarity, this description will refrain from repeating every possiblecombination of the individual steps in an unnecessary fashion.Nevertheless, the specification and claims should be read with theunderstanding that such combinations are entirely within the scope ofthe invention and the claims.

New foot measuring applications for determining the size of feet andappropriate corresponding shoe sizes are discussed herein. In thefollowing description, for purposes of explanation, numerous specificdetails are set forth in order to provide a thorough understanding ofthe present invention. It will be evident, however, to one skilled inthe art that the present invention may be practiced without thesespecific details.

The present disclosure is to be considered as an exemplification of theinvention, and is not intended to limit the invention to the specificembodiments illustrated by the figures or description below. The presentinvention will now be described by referencing the appended figuresrepresenting preferred embodiments.

FIG. 1 illustrates an exemplary process 100 for determining measurementof a foot based on at least one captured image. For example, asdiscussed further herein, process 100 may begin with a step (not shown)of launching an application on a computing device including a camera,such as a smartphone or tablet computer. In some embodiments, process100 may include a step 101 of capturing at least one image of a foot anda horizontal reference object from a first point of view, where a heelof the foot is disposed against a vertical reference object.Furthermore, process 100 may include a step 102 of displaying the atleast one image including the foot, the horizontal reference object, andthe vertical reference object, where one or more camera guides areoverlaid on the at least one displayed image. At least the capturing anddisplaying at steps 101-102 are discussed in detail with respect to FIG.2.

Referring now to FIG. 2, a screen view of the application is depicted.The screen view depicted in FIG. 2 includes a representation of an imagepreview screen of the application, which may be displayed when theapplication is launched on the computing device. In some embodiments,the application indicates four corner alignment guides 11 for thereference paper object 12. In some embodiments, the application willfirst prompt the user to select the reference object from a menu ofchoices. For example, a menu may include options such A4-sized paper orLetter-sized paper. In this case, paper object 12 may represent a pieceof white A4-sized paper. The program may further instruct the user toplace the reference paper with a shorter edge against a wall corner 13.The program next instructs the user to place their foot on the paperwith the heel against the wall. This instruction allows the program todetermine the foot rear limit (e.g., the heel edge) since the actualrear limit will not be visible in the typical viewing angle, such asthat depicted in FIG. 2. This viewing angle will be typical for both auser measuring someone else's foot, and for a user measuring their ownfoot (objects in the screen view may be rotated 180 degrees, dependingon whether the user is measuring their own foot). The program may assumethat the foot rear limit is collinear with wall corner line 13.

In some embodiments, a first step in effectively measuring a target footinvolves establishing the location of the reference paper. Accordingly,the four corners of the reference paper must be established. The camerafield of view may be divided into nine grid blocks, and the user isinstructed to manipulate the camera such that each corner of thereference paper lies within grid blocks 14, 15, 16, and 17. The cornerguides 11 may be displayed for user guidance only, and the program maynot require that the four corners be precisely aligned with theseguides, which may increase usability. The grid blocks, and corner searchin specific blocks, are a key part of the overall measurement algorithmand greatly reduce errors and process time.

In order to find the paper corners within the four grid block areas, theinvention applies aspects of “corner detection” algorithms establishedin the field of computer vision. A two-step process may be utilized todetect corner separation between the paper and the floor. In a firststep, the invention converts the color image data into grey-scale withaverage intensity values. The second step includes downsampling, whichmay include reducing resolution but maintaining key details of the data.

Specifically, during the first step of the algorithm, red-blue-green(RBG) image data is converted into a hue-saturation-value (HSV) colorspace, with intensity values determined based on averaging threedifferent HSV values. In some embodiments, the camera may utilize aBehr-type sensor with double the number of green detection elements thanred and blue elements. Thus, the program may only account for half ofthe green values. In turn, this process ignores and/or removes all colorinformation and converts the data into “brightness” values across alldifferent color values. During the second step, the HSV values are thentransformed into a vector space through computations of first and secondorder differentials to determine velocity and acceleration of change inHSV value. In areas where there is the most change in HSV value, theprogram maintains more data, and where there is less change in HSVvalue, the program increases the rate of averaging of HSV values. This“binarization” process maintains more data along the edges of the paper.The paper corners are then determined based on the intersections of twoedges, and the vertex of a corner is determined as the intersection ofthe two edges at that corner.

In some embodiments, the computing device may be a mobile smartphone ortablet. The image processing described above may be performed based on avideo feed from at least one camera included within or coupled to thedevice. Typically, such devices may generate video at a rate of thirtyframes per second. The corner-detection process described above may thusbe performed on each successive video frame until the program determinesthat (i) four corners have been located, and (ii) there is one corner ineach of grid blocks 14, 15, 16, and 17. In some embodiments, when acorner has been detected within a given block 14, 15, 16, or 17, thereference guide corner 11 in that corresponding grid block will changecolor in the camera preview screen. This color change provides the userwith feedback on the progression of the overall process. If there areproblems at this step (e.g., one or more corners are obscured from thecamera's view, are crumpled or distorted, or if there is no paper in thecamera's view), the program will continue to run the corner detectionprocess on succeeding video frames until a defined time limit isreached. Once such limit is reached, the program may prompt the user toconfirm that all reference paper corners are visible in the view screenof the camera.

Referring back to FIG. 1, process 100 further includes a step 103 of, inresponse to aligning the one or more camera guides with one or more ofthe horizontal reference object and the vertical reference object,determining a measurement of the foot based on the at least one capturedimage from the first point of view. At least the aligning anddetermining at step 103 are discussed in detail with respect to FIGS.2-5. Referring now to FIG. 3, a visualization of the application processis depicted, including measurement algorithm processes of corner andvertex detection, reference establishment, and perspectivetransformation. FIG. 3 illustrates detected corners 18, 19, 20, and 21given the perspective view of the camera. The view in FIG. 3 would nottypically be displayed to a user, but instead, is provided to illustratethe internal processes of the program. In some embodiments, a typicalviewing angle of the camera will likely not be directly over thereference paper, and thus, the paper corners will not appear to besquare ninety-degree corners due to perspective effects. Thesenon-square corners are acceptable for the program as thecorner-detection goal is to determine the corner vertices (vertexpoints) 22, 23, 24, and 25. Next, the program may establish straightlines 26, 27, 28, and 29 between the vertices. After establishing thestraight lines, the program may calculate the angles 30, 31, 32, and 33between the straight lines so that the program can perform a perspectivetransform from a given orientation to a known orientation. By assessingwhether an angle is obtuse or acute, and the respective degrees of theangles, the program can determine the degree to which image data shouldbe corrected for perspective effects. Furthermore, the algorithm of theprogram may error-check that angles 30, 31, 32, and 33 add up to 360degrees.

FIG. 4 illustrates the result of the perspective transformation processdescribed in FIG. 3. The view in FIG. 4 would not typically be displayedto a user, but instead, is provided to illustrate the internal processesof the program. In some embodiments, a rectangle with square angles 34is generated. For example, the program may establish side lengths of thesquared reference frame rectangle based on the known dimensions ofstandard reference paper sizes (e.g., 8.5″×11″ for Letter-sized paper),which are defined in the program. The program may also correct the imagedata within the reference rectangle, including the target foot. In oneembodiment, correction of the image data includes “stretching” the imagedata in order to establish the reference frame having the predefineddimensions of the paper reference 12. In a next step, the program maylook for the foot inside of the reference frame by running an“edge-detection” process similar to the corner-detection processdescribed above. Based on a similar process of image processing andbinarization, the program may establish the edges between the foot 35and the paper inside of the area of the reference rectangle. The edgedetection performed on the foot is more sophisticated than the edgedetection performed on the paper. For example, the edge detectionperformed on the foot may handle minor shadows extending from the mid orrear of the foot. The foot edge detection may iterate on the regioninside the paper using a successively varying gradient threshold todelineate between foot and shadow. Typically, the visual field of thevideo frame will include the leg 36 extending from the foot. The programmay trim the edge-detected object 35 of contents beyond the back edge 37of the reference rectangle. In one embodiment, edge 37 is collinear withwall-floor corner line 13 described above. In turn, this processeffectively trims the leg form 36 and retains the relevant foot sizeinformation. This trimming is appropriate since the user was previouslyinstructed to align the reference paper to the wall-floor corner 13, andto place their heel against this corner.

Referring now to FIG. 5, the program may establish an object frame 38around the foot object 35. The view in FIG. 5 would not typically bedisplayed to a user, but instead, is provided to illustrate the internalprocesses of the program. In some embodiments, the program may determinethe angles 39, 40, 41, and 42 on the object frame to determine thecharacteristic (or major) vector of the frame, and therefore, of thefoot. Next, the program may determine whether the characteristic vectoris orthogonal to the minor axis of the reference frame 34, andtherefore, whether the characteristic vector is orthogonal to the paper.As a result, these algorithmic steps determine if the foot is rotatedrelative to the paper. If the program determines that object frame 38 isrotated beyond a set of preprogrammed dimensional tolerances, then theprogram will transform the image data such that object frame 38 isrotated back to “straight” (e.g., with characteristic vector orthogonalto the minor axis of reference frame 34). At this point, the program isready to start the size analysis of the video frames.

In one embodiment, the program may perform size analysis based on thesize of the object frame 38. For example, length 44 may represent a footlength, and width 45 may represent foot width. These measurements may bedetermined relative to the known measurements of the edges of thereference frame 34. As a first step, the program may perform filteringto disregard results that are too extreme. For example, filtering mayallow the algorithm to be more efficient, as the program does not needto run the rest of the analysis if the initial analysis returns grosserrors. Gross error filtering may include discarding results for length44 or width 45 that are too small (e.g., less than the size of thesmallest infant shoe size). Where the non-conforming results arefiltered out, results for length 44 and width 45 are compared todetermine whether the resulting aspect ratio is within the preprogrammedrange of possibility for a foot. In some embodiments, when the resultsfinish the gross error checking, a visual indication on the camerapreview screen may provide an indication to the user that the foot hasbeen detected in the overall process.

In one embodiment, the program maintains results that are not filteredout and determines a final result based on the average of a preset Xmultiple number of measurements. For example, in some embodiments, fiveresults are averaged. The program will run this final average analysiswhen X consecutive video frames generate measurements, for both length44 and width 45, within a preset tolerance Y of each other. For example,in some embodiments, the tolerance may be set at 5 mm. Depending on theapplication and device used (e.g., depending on camera specifications),in some embodiments, the program may utilize more or less consecutivevalues, may use a greater or lower tolerance than 5 mm, and/or may use adifferent averaging methodology (e.g., accepting any five values withinthe accepted 5 mm tolerance within a ten measurement block of values).

Referring now to FIG. 6, the program may display size information 46within a preview screen. In one embodiment, the program may convertfinal averaged values into shoe sizes based on preprogrammed shoe sizeinformation. For example, a database within the program may maintainsizes and corresponding dimensional values.

In some embodiments, the program may instruct the user to wear a sockbefore stepping on the paper. Although, the processes described abovemay function independent of whether a user is wearing a sock, the use ofa sock in this embodiment may, for example, assist in preventing a footfrom sticking to the reference paper (e.g., due to moisture on thefoot). Furthermore, while running the video analysis, the program mayactivate a built-in flashlight continuously (sometimes referred to as“torch” mode) until a final value is determined. This extra light sourcemay assist to mitigate extreme lighting, such as prominent shadows andglare, that can affect the analysis. For similar reasons, the programmay instruct the user to choose a measurement environment with evenlighting.

In some embodiments, the program may be run locally, without requiringaccess to remote server computations. This client-side calculation maysignificantly improve responsiveness of the application and reducelatency, compared to a solution that would rely on remote server-sidecomputations. In practice, under the proper conditions as describedabove, the analysis may typically be performed by the computing devicein less than five seconds and may be perceived as relatively“instantaneous” to the user.

FIGS. 7A-7D illustrate a representative flow chart of an exemplaryembodiment of the foot measuring application program, depicting thesteps and logic involved in the process. In general, one or more of thesteps described above are represented in this flow chart. As describedabove, the client-side application does not need to access the remoteserver 47 in order to analyze the foot size. In one embodiment, theremote server 47 may be accessed for ancillary functions.

Referring now to FIG. 8, remote server 47 may provide additionalbenefits to the user, such as being able to display informationincluding data relative to past measurement results and population data.For example, such information may be retrieved by accessing server-sidedatabases 48. In one embodiment, remote server 47 is accessed wirelesslythrough internet, cellular, or other wireless networks 49. However, allcore novel foot measuring functionality may occur within the applicationprogram on the client-side computing device 50, as seen in FIGS. 7A-7D.As discussed above, this client-side functionality may significantlyimprove the responsiveness and speed of the application, and may alsoensure that core functionality of the application is available to theuser even when the computing device is not connected to the internet,network 49, or other means of communication. FIG. 8 further illustratesthe core functionality required on the client computing device side insome embodiments, including application 51, camera 52, processor 53,torch 54, and storage device 55. For example, processor 53 may berequired to perform the analysis described in this preferred embodiment,and to run the software application 51 as described. As another example,the local storage 55 may be required to buffer data as part of runningapplication 51, and to store data and results in the event there is nonetwork connection 49 available. In some embodiments, connection toremote server 47 is not a part of the functionality of the application.

Referring now to FIG. 9, in some embodiments, a heel of the foot isplaced against a wall, but the foot is not resting on top of a piece ofreference paper. For example, an alternate reference object 56 of knownsize may be placed next to the foot. An alternate reference object 56may include objects having known sizes, such as credit card 56. Theembodiments discussed with respect to FIG. 9 may function similarly tothose discussed with respect to FIGS. 1-8. For example, reference object56 may correspond to the horizontal reference object at step 101 ofFIG. 1. Furthermore, the program may detect the reference object 56through a similar edge detection process as discussed above, within aprescribed area indicated on the screen, such as the box 57 asdemarcated by the thin white lines. The program may detect the boundaryedge 58 between the wall and floor through a similar process of edgedetection. The program then may then detect the foot in a processsimilar to the embodiments described above, for example, by determininga virtual boundary box around the foot. The program may determine thedimensions of the foot by comparing to the known dimensions of thereference object.

In some embodiments, the method for identifying the foot in the field ofview can be determined by other means. For example, identification maybe based on analysis of skin color compared to the color of thereference paper. Another alternative identification is to detectsubcutaneous blood flow through, for example, “Euler methodology,” inorder to distinguish the skin of the target foot from the referencepaper.

FIG. 10 illustrates an exemplary process 1000 for determiningmeasurement of a foot based on at least one captured image of a verticalreference object. Similar to that discussed with respect to FIG. 1, forexample, process 1000 may begin with a step (not shown) of launching ofan application on a computing device including a camera, such as asmartphone or tablet computer. In some embodiments, process 1000 mayinclude a step 1001 of capturing at least one image of a verticalreference object from a first point of view, wherein at least a portionof a foot is disposed between the vertical reference object and thecamera. At least the capturing and determining steps 1001-1002 arediscussed in detail with respect to FIG. 11.

Referring now to FIG. 11, an embodiment including a computing device 59positioned without a reference object is depicted. In some embodiments,a portion of the foot (such as the heel of the foot as seen in FIG. 11)is placed against a wall, and a computing device 59 (e.g., a“smartphone”) is placed on the floor with the forward-facing surface ofthe device touching the forward-most toe of the foot. In thisorientation, the length of the foot would be measured. Alternatively,the length can be measured with the toe against the wall and device 59placed on the ground against the heel. Furthermore, the width of thefoot can be measured by placing either side of the foot against thewall, and placing device 59 against the opposite side of the foot.

In one embodiment, as illustrated in FIG. 11, no reference object isneeded. In this embodiment, the forward-facing surface of the camera inthis case is defined as the surface with the primary camera, which istypically the surface opposite from the screen. To determine thedimensions of the target foot, the program relies on the focusingfeature of the camera, which determines the distance of the camera fromthe wall through a rangefinding process. Depending on the computingdevice, the user may need to select the wall as the focus subject, forexample by touching on the touchscreen to direct the camera to focus onthe wall area. In addition, the user may need to select the “focus lock”feature of the device, to maintain a focus on the wall, depending on thespecific device.

In order to accurately measure the distance to the wall that bestrepresents the dimension of the foot (either length or width), thecamera may be required to be “square” to the wall, such that theforward-facing camera surface is coplanar with the wall. In oneembodiment, built-in sensors of the device, such as gyroscopes, willdetermine if the device is orthogonal to the ground plane. If the camerais tilted relative to the ground plane, the program will prompt the userto straighten the smartphone, for example, through on-screen directions.To determine if the camera is tilted relative to the wall plane, thedevice may utilize a similar edge detection process, as described in theembodiments above, to determine the wall-to-floor boundary line 60. Theprogram may then determine if this line is angled or horizontal, and mayprompt the user to turn the smartphone if needed in order to achieve ahorizontal line. Once the device orientation is coplanar to the wallwithin preprogrammed tolerances, the distance to the wall is recordedand the program is then able to calculate the foot dimension, and thus,appropriate shoe size. The program may then follow a similar process asin the embodiments described above, for example, by displaying the footdimension onscreen and optionally communicating with remote servers.

In another embodiment, as shown in FIG. 12, without the use of areference object, spatial depth sensors are used. In particular, thisembodiment may be implemented when the application program is running ona mobile computing device that incorporates sensors that measure spatialdepth. Typically, depth-sensing sensors incorporate a methodology knownas time-of-flight infrared laser dot metrology (however, in otherexamples other techniques are possible, such as parallax techniquesusing multiple cameras, for example). Such sensors typically utilize aninfrared emitter that projects a series of infrared dots 1210 ontoobjects in the field of view 1220, as shown in FIG. 12, and at least twocameras that can detect these dots 1210. One or more algorithmsdetermine the depth of the contours and objects in the field of view,based on the time-of-flight of the reflected dot light. This depthinformation allows the device to determine the size and shape of adetected object, based on the calculated absolute depth values. Asrelated to this example, these depth-sensing sensors allow for the sizeand shape of a foot to be determined by the program, and for theresultant foot size results to be displayed as discussed above in theprocess for determining measurement of a foot based on at least onecaptured image of the foot. On some smartphones and mobile devices withone or more depth-sensors on the rear-face of the device, the side ofthe device opposite from the primary device screen facing the user, theuser would direct the device as described above in the process fordetermining measurement of a foot based on at least one captured imageof the foot. For example, the user may direct the device as shown inFIG. 12, and be guided by onscreen prompts. On devices in which one ormore depth-sensors are on the front-face, the side of the phone facingthe user, the user must turn the primary device screen away towards thefoot being measured in order to utilize the program to size the foot. Inthese cases, the program may guide the user by providing indications tothe user of a measurement completion through other device means, such asan audible tone or a brief flashing of the rear-side flash. The userwould then be able to turn the device screen back towards themselves inorder to see the measurement result.

Although a variety of examples are used herein to describe embodimentswithin the scope of the appended claims, no limitation of the claimsshould be implied based on particular features or arrangements in suchexamples. Accordingly, one of ordinary skill in the art will be able touse these examples to derive a variety of implementations. Althoughsubject matter may have been described in language specific to examplesof different structures and processes, it is to be understood that thesubject matter defined in the appended claims is not necessarily limitedto these structure and processes. For example, such functionality can bedistributed differently or performed in components other than thoseidentified herein. Rather, the features are disclosed as examples ofcomponents of systems and methods within the scope of the appendedclaims.

What is claimed is:
 1. A method for measuring an object, comprising:capturing, by a mobile computing device, at least one image of a footand a plurality of reference dots from a point of view; displaying, bythe mobile computing device, the at least one image comprising the foot,a vertical reference object, and the plurality of reference dots,wherein one or more camera guides are overlaid on the display of the atleast one image; aligning, by the mobile computing device, the one ormore camera guides with the foot and the vertical reference object; anddetermining, by the mobile computing device, a measurement of the footbased on the at least one image and the plurality of reference dots fromthe point of view.
 2. The method of claim 1, wherein the verticalreference object comprises a wall.
 3. The method of claim 1, furthercomprising: generating, by the mobile computing device, a squaredreference frame based on the at least one image; and performing, by themobile computing device, edge detection on the squared reference frameto establish edges of the foot.
 4. The method of claim 3, furthercomprising utilizing, by the mobile computing device, a successivelyvarying gradient threshold to delineate between the foot and at leastone shadow associated with the foot.
 5. The method of claim 1, whereindetermining a measurement of the foot further comprises: performing, bythe mobile computing device, edge detection on the at least one image todetect a foot object; generating, by the mobile computing device, anobject frame based on the detected foot object; and performing, by themobile computing device, a size analysis based on a size of the objectframe to obtain at least one measurement.
 6. The method of claim 5,wherein the at least one measurement e comprises at least one length ofthe object frame and at least one width of the object frame.
 7. Themethod of claim 1, wherein a portion of the foot is disposed directlyagainst the vertical reference object.
 8. The method of claim 1, whereinthe plurality of reference dots comprises a series of infrared dotsprojected into at least a portion of the point of view.
 9. The method ofclaim 1, further comprising: projecting, by a depth sensor of the mobilecomputing device, the plurality of reference dots; and detecting, by atleast one camera in the mobile computing device, at least a portion ofthe plurality of reference dots.
 10. The method of claim 1, furthercomprising determining the measurement of the foot by: determining, bythe mobile computing device, a depth of contours of the foot in theimage of the foot based on a time-of-flight measurement determined basedon at least a portion of the plurality of reference dots; calculating,by the mobile computing device, at least one absolute depth value basedon the depth of contours of the foot; and calculating, by the mobilecomputing device, a size and a shape of the foot based on the absolutedepth value.
 11. A mobile computing device, comprising: a processor; acamera; a memory storing instructions that, when executed by theprocessor, cause the mobile computing device to: capture, using thecamera, at least one image of a foot and plurality of reference dotsfrom a point of view; display the at least one image comprising thefoot, a vertical reference object, and the plurality of reference dots,wherein one or more camera guides are overlaid on the display of the atleast one image; determine an alignment of the one or more camera guideswith the foot and the vertical reference object; and determine ameasurement of the foot based on the at least one captured image and theplurality of reference dots from the point of view.
 12. The mobilecomputing device of claim 11, wherein the vertical reference objectcomprises a wall.
 13. The mobile computing device of claim 11, whereinthe instructions, when executed by the processor, further cause themobile computing device to: generate a squared reference frame based onthe at least one captured image; and perform edge detection on thesquared reference frame to establish edges of the foot.
 14. The mobilecomputing device of claim 13, wherein the instructions, when executed bythe processor, further cause the mobile computing device to utilize asuccessively varying gradient threshold to delineate between the footand at least one shadow associated with the foot.
 15. The mobilecomputing device of claim 11, wherein the instructions, when executed bythe processor, further cause the mobile computing device to determine ameasurement of the foot by: performing edge detection on the at leastone captured image to detect a foot object; generating an object framebased on the detected foot object; and performing a size analysis basedon a size of the object frame to obtain at least one measurement. 16.The mobile computing device of claim 15, wherein the at least onemeasurement comprises at least one length of the object frame and atleast one width of the object frame.
 17. The mobile computing device ofclaim 11, wherein a portion of the foot is disposed directly against thevertical reference object.
 18. The mobile computing device of claim 11,wherein the plurality of reference dots comprises a series of infrareddots projected into at least a portion of the point of view.
 19. Themobile computing device of claim 11, wherein: the mobile computingdevice further comprises a depth sensor; and the instructions, whenexecuted by the processor, further cause the mobile computing device to:project, using the depth sensor, the plurality of reference dots withinthe point of view of the foot; and detect, using the camera, at least aportion of the plurality of reference dots.
 20. The mobile computingdevice of claim 11, wherein the instructions, when executed by theprocessor, further cause the mobile computing device to determine themeasurement of the foot by: determining a depth of contours of the footin the image of the foot based on a time-of-flight measurementdetermined based on the plurality of reference dots; calculating atleast one absolute depth value based on the depth of contours of thefoot; and calculating a size and a shape of the foot based on theabsolute depth value.